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Abstract 
Few studies have considered academic vocabulary interventions for students who have learning 
disabilities (LD) and also are English Learners (EL). This research explored the effects of the 
third year of the CHAAOS (Creating Habits that Accelerate the Academic Language of 
Students) vocabulary intervention for eighth grade students who received English Language Arts 
(ELA) in special education classes. Over 70% of student participants had LD and over half were 
EL. Special education teachers delivered 12 weeks of intervention spaced across seven months 
to 36 students. Researchers compared outcomes on receptive and expressive measures with 17 
students in business-as-usual (BAU) special education ELA. Students receiving CHAAOS 
intervention learned the 48 taught words better than students in BAU conditions, retained their 
knowledge of words four weeks following the close of the intervention, and ended the year with 
higher vocabulary scores on a standardized measure than students in BAU classes. Students who 
were EL performed similarly on receptive knowledge of words to students who were native 
English speakers. This study documents the effectiveness of explicit vocabulary routines for 


students who have LD and are also EL. 
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Continuing CHAAOS: Vocabulary Intervention for Students with Disabilities in Eighth 
Grade Who Are Also English Learners 

As students approach their adolescent years, understanding meanings of academic words 
takes on increasingly large proportions of variance in reading comprehension (Catts et al. 2006; 
Hjetland et al., 2019; Holahan et al., 2018). Students who struggle with academic language and 
reading comprehension have difficulty succeeding in content area courses in secondary grades 
(Foorman et al., 2015). Unfortunately, learning new vocabulary—especially the academic 
vocabulary of books and content areas—is more difficult than learning to decode words. Unlike 
decoding skills, which have limited and specific sets of relations and rules, vocabulary words 
used in texts continue to expand in number and complexity throughout schooling (Paris, 2005). 

Learning meanings of new words is especially challenging when reading ability is poor, 
because most words are learned through repeated exposure in text (Nagy & Townsend, 2012), 
and poor readers read less text than their average-reader peers (Ahmed et al., 2016; Jenkins et al., 
2003). Students with learning disabilities (LD) often have extraordinary difficulty with the 
vocabulary words in texts because these academic words are not commonly used in conversation 
and instead appear in texts these students may be unable to read independently. Students who are 
English Learners (EL) have these challenges also, coupled with lack of exposure to academic 
language in English at home (Hart & Risley, 1995; Kieffer, 2010), which makes language gaps 
difficult to fill as the complexity of language used in school increases (Proctor et al., 2005). 

For students who have LD and may also be EL, improving understanding of academic 
language can be arduous due to the sheer volume of word meanings to learn. Differences in 
language comprehension between students with average and below-grade level vocabulary can 


be huge by the time students reach middle school regardless of first language (Lesaux & Harris, 
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2013). Research consistently shows students who struggle with reading acquisition and 
vocabulary can learn meanings of new words in the short term on researcher-developed measures 
(Jitendra et al., 2004; Kuder, 2017). A few studies in middle school have also documented 
retention of taught words over a few weeks or months (Bos & Anders, 1990; O’Connor et al., 
2019). Nevertheless, for students with LD, it remains rare to demonstrate generalized 
improvement in academic vocabulary and comprehension on standardized reading measures 
(Vaughn et al., 2019). The present study reports findings from the third year of a three-year 
project in which academic vocabulary was taught to students with LD, including students who 
were EL, in special education English/Language Arts (ELA) classes in middle school. 
The Role of Vocabulary in Reading Comprehension 

Teaching vocabulary effectively to poor readers in middle school is no small challenge, 
because poor readers are hampered by serious delays in reading words, understanding word 
meanings, and comprehension compared to peers, who mastered the reading basics years before 
(Swanson & Deshler, 2003). We focus on academic vocabulary because of its strong theoretical 
role in comprehension and potential instructional accessibility through its emphasis on oral 
language, as well as on reading. Most theories of comprehension include vocabulary as an 
essential component. As examples, the much studied Simple View (Gough & Tunmer, 1986) 
posits reading comprehension as a function of the ability to read words (i.e., decoding and 
recognizing words) and language comprehension (i.e., vocabulary and language syntax). A 
recent study by Hjetland et al. (2019) found that these two factors accounted for 99% of reading 
comprehension among the nine-year-old participants. Kintsch (1988) acknowledges the 
importance of vocabulary, but complicates the act of comprehending language and text with the 


process of using words and syntax to generate a mental representation of meanings of phrases, 
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sentences, and paragraphs as the listener or reader moves through text. His construction- 
integration model of comprehension emphasizes actions of the reader in constructing meaning. 
Perfetti and Stafura (2014) accept both of these views of reading comprehension, but focus on 
depth of word knowledge through lexical representation of words, including ways in which word 
meanings incorporate their graphical and semantic connotations, as well as linkages to contexts 
and usage. In all of these theoretical models, understanding word meanings plays a fundamental 
role in reading comprehension. 

The essential nature of word meaning in comprehension for older students is exemplified 
in studies such as Cromley and Azevedo (2007) and Cromley et al. (2010), who examined the 
effects of vocabulary on reading comprehension of students in ninth grade and beyond and found 
vocabulary to be a significant direct and indirect predictor of comprehension. More recently, 
Ahmed and colleagues (2016) studied multiple predictors of reading comprehension among 
middle to high school students using a latent variable structural equation model. Both supporting 
and extending the earlier work of Cromley and Azevedo, results indicated that the effect of the 
latent construct of vocabulary was related significantly to reading comprehension across grades 7 
to 12 (B = .35-.51). Similarly, Holahan et al. (2018) assessed the relation between verbal ability 
and passage comprehension using multilevel growth models. Their analysis indicated that 
vocabulary accounted for unique variance in passage comprehension (f = .13). These findings 
reinforce treating vocabulary as a relevant instructional target for students in middle school. 
Vocabulary Instruction 

Several research reviews have identified key practices associated with strong intervention 
effects for students who have LD (Jitendra et al., 2004; Swanson & Deshler, 2003) and for 


students who are EL (Hall et al., 2017; Jozwik & Douglas, 2017; Truckenmiller et al., 2019). 
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Among these practices, explicit instruction is championed extensively (e.g., Hughes et al., 2017; 
McLeskey, et al., 2017). Hughes et al. characterize explicit instruction as “unambiguous, 
structured, systematic, and scaffolded” (p. 140). When compared with other approaches to 
teaching vocabulary, Jenkins et al. (1989) and Bos and Anders (1990) found explicit instruction 
to be more effective; however, these studies had only a few instructional sessions, so long-term 
effects were not evaluated. 

Explicit vocabulary instruction incorporates a purpose and rationale for learning 
vocabulary, clear explanations and modeling, guided practice with feedback, ample practice 
opportunities, and active participation from students (Archer & Hughes, 2011). Academic words 
and definitions are taught directly to students, examples and nonexamples are incorporated for 
practice, multiple exposures and practice opportunities are provided, and the academic words are 
used within exercises that require meaningful interaction with academic words. 

Several studies have demonstrated the success of explicit vocabulary instruction in 
middle school for students in general education classes. For example, Lesaux et al. (2010) 
developed the Academic Language Instruction for All Students (ALIAS) vocabulary intervention 
for English/Language Arts (ELA) classrooms that were low-performing and had high numbers of 
EL students. Results indicated that intervention students improved their knowledge of taught 
words and the intervention benefited both EL and Native English Speakers (NES). Additionally, 
McKeown et al. (2018) studied the effects of a 6" and 7" grade vocabulary intervention, Robust 
Academic Vocabulary Encounters (RAVE), for middle school students. Across both years of the 
study, RAVE students demonstrated growth on their knowledge of taught words when compared 
to control students; however, these studies did not include students with disabilities. 


Peer interaction and dialogue is another of the practices common to researched 
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recommendations for both EL and students with LD (Jozwik & Douglas, 2017; Truckenmiller et 
al., 2019). The early research of Bos and Anders (1990) demonstrated that students with LD 
placed in interactive vocabulary learning conditions scored significantly higher on 
comprehension than their peers in a definition-only condition. Lawrence et al. (2015) found that 
classroom discussion quality predicted vocabulary outcomes in particular for poor readers, and 
rich discussions have been found useful for students with disabilities (Englert & Dunsmore, 
2004) and other poor readers (Kamil et al., 2008, Scammacca et al., 2007). Interestingly, 
recommendations from research for improving vocabulary of students who are EL and for 
students with disabilities are remarkably similar (Crosson et al., 2019; Elleman et al., 2019). 

The combination of explicit instruction and peer interactions for practice have been used 
extensively in vocabulary studies in general education environments (e.g., Carlo et al., 2004; 
Jones et al., 2019; Lawrence et al., 2015; Lesaux et al., 2010; McKeown et al., 2018). Some of 
these general education studies have included students who have LD (Jones et al.; Lawrence et 
al.) or who are EL (August et al., 2014; Carlo et al.; Lesaux et al., Truckenmiller et al., 2019). 
Studies that focus on students with disabilities who may also be EL are rare, and studies that do 
include them rarely report results for these groups separately (e.g., Jones et al., 2019). 

Other studies that included students with LD or who were EL have focused on content 
area words in social studies (e.g., O’Connor et al., 2017; Swanson et al., 2016) or science 
(August et al., 2014; Helman et al., 2015; Kennedy et al., 2015; Seifert & Espin, 2012). Because 
the goal of those studies was to improve content area learning, the interventions included many 
aspects of reading comprehension in addition to vocabulary, and so vocabulary effects could not 
be isolated to vocabulary instruction specifically. 


Truckenmiller et al. (2019) synthesized outcomes for academic language instruction for 
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students who were EL; however, if studies included more than 30% of students eligible for 
special education, they were excluded in the analysis. Nevertheless, Truckenmiller et al. 
identified similar attributes to effective instructional models as found in studies of special 
education: a small set of words not used commonly in everyday speech, multiple exposures 
across relevant contexts, multimodal methods such as graphics displaying word meanings, and 
discussion with peers and teachers, recommendations echoed by Crosson et al. (2019) and Kuder 
(2017). Unfortunately, these reviews revealed few impacts of the instruction on standardized 
measures, replicating findings of Jitendra et al. (2004) and Scammacca et al. (2007). 
Context for the Current Study 

Few studies of academic vocabulary instruction have been conducted in middle school 
special education settings that include students who have LD and may also be EL, which is 
concerming due to increasing numbers of students with LD who are also EL in schools 
(Roderiguez & Rodriguez, 2017). The current study explores the effects of the third year of the 
Creating Habits that Accelerate Academic Language of Students (CHAAOS) vocabulary 
intervention taught by special education teachers to students with disabilities in their intact 
English/Language Arts (ELA) classes. Special education teachers of ELA were assigned 
randomly to either CHAAOS or Business as Usual (BAU) conditions in Year 1 of the study and 
were trained to implement 15 minutes of explicit vocabulary intervention with their sixth grade 
students. Teachers taught meanings of 48 grade-appropriate academic words (O’Connor et al., 
2019). In seventh grade, with a mix of new and returning students and teachers, students were 
taught an additional 48 academic words (O’Connor et al., in press). In eighth grade, a new set of 
48 academic words were taught and these results are reported here. In Years 1 and 2, students in 


CHAAOS classes learned and maintained the age-appropriate words significantly better than 
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those in BAU conditions (O’Connor et al., 2019, in press). 

Our research questions were: What are the effects of the 8" grade CHAAOS intervention 
(1) on learning the taught vocabulary? (2) on using the taught vocabulary in contexts? (3) on 
maintaining knowledge of taught vocabulary? (4) on standardized measures of vocabulary and 
comprehension? and (5) How does first language affect response to the CHAAOS intervention? 

Method 

The third year of CHAAOS instruction was designed for eighth graders with disabilities 
who may also be EL. As in many studies of academic language instruction, we used Coxhead’s 
Academic Word List (2000) as our first resource; however, we also cross-referenced the words 
we selected with words recommended in Biemiller’s Words Worth Teaching (2010), sixth 
through eighth grade academic word lists from the Common Core State Standards (National 
Governors Association, 2010), and the school district’s list of words to teach in 8" grade ELA. 
From these resources, we selected 48 words for instruction during eighth grade. These 48 
academic words were taught across three 4-week cycles during the school year with each cycle 
of instruction including 16 words. 
Participating Teachers and Students 

Our study took place in one southern California school district that enrolled 

approximately 20,000 students. The district had a high proportion of socioeconomically 
disadvantaged students (81%) and English learners (36%) enrolled. We worked with three 6" 
through 8" grade middle schools with enrollments that ranged from 973-to-1404 students and 
these schools participated in all three years. 
Teachers 


Four eighth grade special education teachers who had taught students with disabilities for 
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4-to-29 years participated in this study. Their assignment to condition was based on Year 1, in 
which the sixth grade teacher in each of the three schools was assigned randomly to implement 
CHAAOS or serve as a BAU class, in essence resulting in random assignment of schools to 
condition. Personnel in the BAU school were invited to use CHAAOS materials in subsequent 
years. In Years 2 and 3, schools maintained their assignments to condition with the participating 
grade. 

One of the CHAAOS schools added an additional special education teacher in seventh 
and eighth grades. Thus three teachers taught in the CHAAOS condition in the current study. 
The fourth teacher taught in the BAU condition in eighth grade (the current study); however, he 
taught in the CHAAOS condition with his sixth and seventh grade classes, which were not part 
of the current study. Because each grade had its own set of materials, we found no crossover 
between his sixth and seventh grade CHAAOS and eighth grade BAU classes. Two additional 
teachers had only one or two students who had been in the BAU condition in previous years and 
maintained their BAU status as eighth graders. 

Students 

As this was a multi-year project, it is helpful to know the historical participation of the 53 
eighth graders who participated in this third year of the project (CHAAOS n = 36; BAU n =17). 
Of our 36 CHAAOS students, 8 began CHAAOS intervention in sixth grade, 20 started 
intervention in seventh grade, and 8 began intervention in eighth grade. Of the 17 BAU students 
in Year 3, 12 were in the BAU class in sixth grade, one joined the BAU in seventh grade, and 
four joined the BAU in eighth grade. Due to attrition and new students enrolling in the study, we 
focus on the 8" grade cohort rather than cumulative effects of intervention. 


All students received special education services and were eligible based on the following 


Vocabulary Intervention 11 


designations: Specific Learning Disability (76%), Other Health Impairment (9%), Autism 
Spectrum Disorder (8%), Speech/Language Impairment (4%), and Emotional Disturbance (2%). 
Most students were EL (72%) with the primary language indicated as Spanish in school records. 
CHAAOS Intervention 
CHAAOS eighth grade words were clustered into three cycles. Each cycle included 16 
lessons that were taught over 4 weeks. Teachers were asked to teach 4 lessons per week for 
approximately 15 minutes each. The core instructional components of CHAAOS lessons were 
based on explicit instruction and included: 
e Introduction of 4 new academic words per week 
e Explicit instruction using student-friendly definitions 
e Pairing definitions with a student-friendly context to illustrate meaning 
e Follow-up vocabulary activities involving teacher guided, peer practice, and independent 
practice of words in context using scenarios, photographs, and graphic organizers 
e Multiple practice opportunities with immediate feedback 
To encourage peer interaction, we used visual depictions of meanings in everyday 
contexts through drawings, photographs, and graphic organizers discussed with students as 
teachers displayed PowerPoint slides. Teachers followed whole-group instruction with partner 
and small group activities that encouraged students’ use of the words in discussion and brief 
writing activities. Although instruction was primarily oral, words were also shown in print on 
the projected slides, on word cards used in peer practice, and on writing activities in student 
workbooks developed for the study. In these ways, CHAAOS integrated phonemic and 
graphemic forms of words with common uses for the words and practice pronouncing and using 


the words with teachers and peers. 
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Teachers in each CHAAOS classroom received instructional materials that included a 
teacher manual, 48 PowerPoint lessons divided into three cycles, student notebooks, and 
supplemental activities. The teacher manual was a hard copy of all lessons with directions and 
suggested scripts to use while teaching. The PowerPoint lessons were designed to be used with 
the entire class while providing instruction. The student notebooks mirrored the PowerPoint 
lessons to allow students to follow along and write responses. Supplemental activities were 
prepared for teachers and included individual flashcards sets of the academic words for students 
to use in class and a class review game of either BINGO or Jeopardy to be used as the 
culminating activity. 

To ease teacher implementation, each week followed a predictable routine for introducing 
and contextualizing the four new words per week across the 3 cycles. On Mondays teachers 
introduced the four new words and discussed meanings with students. On Tuesdays and 
Wednesdays, teachers reviewed all four words and focused instruction and peer activities on two 
of the four words. Instruction on Thursdays integrated the four words with previously taught 
words with peer activity for practice and expansion of contexts. A sample instructional week of 
PowerPoint slides is shown in Appendix A. All of these materials are available to download and 


print for free at: https://gsoe.education.ucr.edu/CHAAOS/index.php 


Teachers and researchers met at the start of the 8 grade school year to review procedures 
and participants. Cycle 1 instruction was provided in August and September, Cycle 2 instruction 
in October and November, and Cycle 3 instruction in January and February. During the 
instructional cycles, the research team completed observations in the classrooms to document 
fidelity to the CHAAOS activities. During remaining class time in the ELA classes, one of the 


CHAAOS teachers used the same reading programs used in the BAU class, described below. 
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The other two CHAAOS teachers focused primarily on writing and editing skills during class 
time not spent on CHAAOS, teachers in both conditions used Newsela on Fridays. 
Instruction in the Business as Usual Class 

The BAU teacher used two reading programs in his class. Four days per week he 
implemented Corrective Reading, Decoding C (Englemann et al., 1988). Lessons began with 
decoding new words in the day’s reading, review of previously taught words, and introducing 
one or two new vocabulary words. The majority of class time was spent reading aloud the day’s 
story individually or chorally, which has controlled text to practice decoding patterns introduced 
sequentially. The teacher inserted questions at prescribed times and discussed answers when 
varied responses were appropriate. The last 5 minutes students paired up for fluency practice, 
starting from the beginning of the day’s story. 

On Fridays, the teacher reported using Newsela, a web-based program used by all four 
teachers with adjustable reading levels implemented through Chrome Books. This activity was 
not observed in the BAU; however, CHAAOS classes also used this program weekly. 
Observations in Classes 

Our observation tool used a 1-to-3 point rating for the four core activities within the 
lesson: Discussing words and definitions, Providing multiple contexts, Practice/peer activities, 
and Providing student support (see Appendix B). Each main activity was subdivided into further 
components. For example, Discussing words and definitions included how the teacher introduced 
and reviewed words and whether the teacher provided repetitions of the words and opportunities 
for students to respond to questions and offer examples. Overall, there were 12 areas where 
teachers could receive a 1-3 rating. Several of the observation tool features could be common 


across conditions, including minutes spent on vocabulary instruction, modeling, guided practice, 
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independent practice, scaffolding student responses, and corrective feedback. 

Inter-rater reliability during observations was established in Year 1 at 92% across 8 joint 
observations (see O’Connor et al., 2019). In Year 2, inter-rater reliability was established 
between two raters again with 92% percent agreement on eight observations (O’Connor et al., in 
press). Inter-rater reliability in Year 3 found 91% agreement over three observations. In 
CHAAOS classes, three observations were conducted in Cycle 1 for each teacher and two in 
Cycles 2 and 3, for a total of seven observations per teacher. In the BAU class, two observations 
were conducted per cycle for a total of six observations, which are described in the results. 
Measures 

A set of experimenter-designed measures of vocabulary were administered pre- and post- 
instructional cycle on taught words. The first assessed word knowledge through a paper-and- 
pencil multiple-choice assessment and an individual oral assessment. The second involved how 
well students were able to use words in context, measured through a paper-and-pencil assessment 
and an individual oral assessment. Following these cycle specific tests, measures of maintenance 
of taught words and standardized measures of reading and vocabulary were collected 4-weeks 
after the conclusion of the intervention. 

Word Knowledge 

We assessed word knowledge through an experimenter designed 16-item multiple-choice 
test of taught words that was given pre and post-cycle. Students were asked to circle the correct 
definition for the taught word among 4 multiple-choice options. Test-retest reliability ranged 
from .88 to .96 across cycles. Correlation coefficients between the multiple-choice measure and 
the Comprehensive Receptive and Expressive Vocabulary Test (CREVT) were significant (r = 


522, p = 001). 
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Using Words in Context 

We assessed how students used vocabulary words in context with an experimenter 
designed 8-item paper-and-pencil measure that required students to use the taught vocabulary 
words at the sentence and passage level, pre- and post-instruction at each cycle. Test-retest 
reliability ranged from .88 to .96 across cycles. Estimates of validity were obtained for the usage 
measures with the CREVT (r = .47, p = .003) and Woodcock-Johnson Passage Comprehension 
(r = .39, p = .014). 
Expressive Vocabulary 

To assess how well students could express their knowledge of vocabulary words students 
were administered individually an expressive test where students were asked to generate a 
definition and sentence for the taught vocabulary words. Similar to others (Cain et al., 2004), we 
developed a 3 point scale (0 = incorrect, 1 = partially correct, 2 = correct) to describe the quality 
of student-generated definitions and sentences for taught words. Estimates of validity were 
obtained for expressive vocabulary scores with the CREVT (r = .52, p = .000) and the WJ 
passage comprehension (r = .46, p = .000). 
Maintenance 

The three cycles of instruction were taught between August and February. Similar to the 
studies of CHAAOS in 6" and 7" grade, we were interested in whether CHAAOS students 
maintained knowledge of the 8th grade words they had learned four weeks after the conclusion 
of the intervention. To test maintenance of vocabulary learning, we developed an 18-word 
vocabulary measure that drew six words from each of the three cycles. We selected the six words 
on which students scored lowest at pretest and highest on the immediate posttest, thus indicating 


words on which students showed the strongest growth in word knowledge for that cycle. 
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Correlation coefficients between the maintenance measure and the three multiple choice 
measures at each cycle were significant (r = .82, .87, .85; p = .001). Additionally, the 
maintenance measure correlated significantly with the CREVT (r = .31, p = .020) and WJ 
Passage Comprehension (r = .362, p = .006). 

Comprehensive Receptive and Expressive Vocabulary Test, 3' ed. (CREVT; Wallace & 
Hammill, 2013). 

The CREVT is an individually administered standardized measure of vocabulary and was 
administered to participants during the spring of their eighth grade school year. Receptive 
vocabulary requires students to point to a picture that represents what the examiner says. 
Expressive vocabulary is measured by asking students to define a word. Internal consistency 
across subtests is high, with coefficient alphas of .85-.96. The general vocabulary score, which 
combines results from the receptive and expressive scores, is reported here. None of the words 
on this measure were taught in CHAAOS lessons. This measure was also used at the beginning 
and end of sixth grade, and end of seventh grade, and these scores were reported in earlier papers 
(O’Connor et al., 2019). 

Woodcock-Johnson Tests of Achievement IIT (WJ-III; Woodcock et al., 2001). 

The reading comprehension subtest of the WJ-HI was administered individually to 
students during the spring of eighth grade. Passage Comprehension was measured by asking 
students to independently read short sentences that increase in difficulty that have a missing 
word; students are asked to supply the missing word. As with the CREVT, the WJ-III was also 
administered when students were in sixth and seventh grade, reported previously (O’Connor et 
al., 2020). 


Results 
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Prior to analyzing data to answer our research questions, we determined the fidelity of 
implementation for the intervention. Next we analyzed the effects of the vocabulary intervention 
on knowledge of taught words. Then, we compared performance of CHAAOS and BAU students 
on how well they could use the taught words in context for each cycle. Next we analyzed scores 
on the maintenance test to determine if there was evidence of maintaining knowledge of words 
over time. Next, we were interested in whether standardized scores changed for students. Lastly, 
we compared performance of EL in the intervention with NES who also received intervention. 
Fidelity of Implementation 

The three CHAAOS teachers were consistent in their implementation and their overall 
fidelity was 88% (i.e., 2.64 of 3 points possible per item). In at least half of our observations, 
teachers varied the types of responses from students as directed in the manual, such as whole 
class, choral, individual, or group practice. Teachers often drew on student experiences to 
support their understanding of the words and provided additional explanations, examples, and 
sentences. Teachers were consistent with providing feedback, compliments, and positive 
reinforcement. When scaffolding was required, teachers provided individual and group support 
and discussion to make sure students understood the sentences and images that were presented. 
Additionally, teachers often rephrased a student response to shape the student’s words 
appropriately and prompted students with guided questions. On average, teachers implemented 
CHAAOS for 19 minutes per observation. 

We observed in the BAU class for about 20 minutes each visit, usually in the first part of 
class in which the teacher provided direct instruction on decoding, guided student practice, 
introduced one or two new vocabulary words, and began directing reading of the day’s story. 


Across the six observations, vocabulary instruction ranged from 1 to 4.6 minutes, and averaged 
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2.1 minutes. Some of the elements of explicit vocabulary instruction were present; the teacher 
provided a user-friendly definition, encouraged students to repeat the definition, and when the 
word was encountered in text, asked students to provide the definition again. Unlike CHAAOS 
classes, no additional contexts for word usage were observed, nor did students practice the 
definitions with peers or generate usage for the taught words. 

On the items related to student support, the BAU teacher did not differ from CHAAOS 
teachers, averaging scores of 90% (2.71 of 3). He praised student effort enthusiastically, pointed 
out parts of words that were partially correct, scaffolded student responses by asking direct 
questions to guide students toward inferences, and rephrased students’ answers. The CHAAOS 
word list had been provided to all participating teachers and their administrators, including those 
in the BAU, and several of the CHAAOS words were included in the eighth grade district 
vocabulary list; however, we did not observe any of the CHAAOS words being taught in the 
BAU class. 

Analytic Decisions 

We were interested in whether group differences would be found in performance on 
multiple measures of vocabulary during instructional cycles 1-through-3. Multivariate analysis of 
covariance (MANCOVA) allowed us to examine group differences on several dependent 
variables simultaneously while also controlling for Type 1 error and taking into account the 
interrelations among variables (Raykov & Marcoulides, 2008). Considering our pretest measures 
were administered immediately prior to instruction and correlated significantly with our posttest 
measures (r = .346-.366, p =.004-.007), we decided to use pretest scores as covariates. 

Word Knowledge and Usage Measures 


MANCOVA was conducted for the multiple-choice and usage measures using pretest 
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scores as covariates. Descriptive statistics are shown in Table 2. Treatment had a significant 
effect on vocabulary word knowledge and usage, A = 0.305, F(6, 40) = 15.001, p = .000, np? = 
0.692. Pillai’s Trace, Hotelling’s and Roy’s tests all converged with Wilk’s Lambda yielding 
significant results. Examination of ANCOVA results revealed significant effects for vocabulary 
word knowledge at all cycles (Cycle 1: F(1, 45) = 71.550, p = .000, np* = 0.614; Cycle 2: F (1, 
45) = 79.628, p = .000, np* = 0.639; Cycle 3: F(1, 45) = 32.195, p = .000, np* = 0.417). 
ANCOVA results for the vocabulary usage measures were mixed. The results were not 
significant at Cycle 1, F(1, 45) = 2.854, p = .098, np* = 0.060. However, results were significant 
at Cycles 2 and 3 (Cycle 2: F(1, 45) = 5.918, p = .019, np* = 0.116; Cycle 3: F(1, 45) = 4.490, p 
= .040, np* = 0.091). 
Expressive Measures 

MANCOVA was also conducted for the student generated definitions and sentences 
across cycles using pretest scores as covariates. Descriptive statistics are shown in Table 2. 
Treatment had a significant effect on expressive vocabulary, A = 0.643, F(6, 39) = 3.613, p = 
.006, np” = 0.357. For student generated definitions, ANCOVA results were significant at all 
cycles (Cycle 1: F(1, 44) = 20.812, p = .000, np” = 0.321; Cycle 2: F (1, 44) = 11.619, p = .001, 
Np = 0.209; Cycle 3: F(1, 44) = 5.557, p = .023, np* = 0.112). For student generated sentences, 
ANCOVA results were once again mixed. Results were significant at Cycles 1 and 2 (Cycle 1: 
F(1, 44) = 8.699, p = .005, np” = 0.165; Cycle 2: F(1, 44) = 5.530, p = .023, np? = 0.112). 
However, results were nonsignificant at Cycle 3 (p =.098). 
Maintenance 

To determine how well CHAAOS and BAU students remembered definitions of taught 


words four weeks after completion of the intervention, we conducted repeated measures 
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ANOVA, which compared performance of CHAAOS and BAU at pretest, immediately after 
instruction, and at maintenance. A significant effect was found for the interaction of treatment 
and time, F(2, 90) = 53.67, p = .000, Np = 0.544. Although no difference was found between 
CHAAOS and BAU students at pretest (M =5.81 and 6.68, respectively; p = .257), students 
differed at immediate posttest (M =16.26 and 7.50, p = .000) and maintenance (M =13.97 and 
7.23, p = .000) with CHAAOS students scoring significantly higher than students in BAU. 
Standardized Measures of Vocabulary and Reading Comprehension 

Of particular interest was whether CHAAOS and BAU students differed on standardized 
measures of vocabulary and comprehension, which were gathered when students first entered the 
study and following treatment in Year 3. ANCOVA results indicated that treatment had a 
significant, positive effect on posttest vocabulary scores on the CREVT, F(1,41) = 6.45, p = 
.015, np* = 0.136, after controlling for pretests. However, treatment did not have a significant 
effect on posttest comprehension scores on the WJ-III after controlling for pretest differences, 
F(1, 41) = 1.55, p = .221. Table 3 shows means and standard deviations on these measures. 
Exploratory EL and NES Performance 

Because we were interested in whether students who were EL and NES differed on 
learning CHAAOS vocabulary words, we analyzed the performance of EL and NES within the 
treatment group only. ANCOVA results found no significant differences on posttest scores on 
the multiple-choice test of vocabulary knowledge (p = .547) or usage measures (p =.525) after 
controlling for pretest differences. 

The group size was small for this analysis, and so lack of difference should be interpreted 
cautiously. Examination of total scores across cycles for students who were EL and NES in the 


CHAAOS condition in Table 4 lend support to the notion that students learned definitions of 
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words receptively similarly regardless of first language; however, expressive performance may 
have been more difficult for students who were EL. 
Discussion 

Despite growing numbers of students with LD who are also EL (Rodriguez & Rodriguez, 
2017), few studies of vocabulary intervention have focused on this population. Results of the 
current study show that students who received 12 weeks of the intervention displayed consistent 
and substantial improvements on the multiple-choice vocabulary measure, in which effect sizes 
were large across cycles (np* = 0.56). This positive effect was mirrored among our expressive 
measures, with an average larger effect for generating definitions (np” = 0.21) than for generating 
sentences (np* = 0.14). These findings, which replicate the first and second year of the 
intervention (O’Connor et al., 2019, in press), also align with prior research that has found that 
teaching academic words directly leads to improved word learning for students with LD (Jitendra 
et al., 2004) and for students who are EL (Truckenmiller et al., 2019), even though the words we 
taught were several grade levels above students’ independent reading ability. 

We share the concern of other researchers that reading instruction in middle school is 
often eliminated in favor of tutoring support for passing courses (Bulgren et al., 2013). Unlike 
other studies, our observations in BAU classes revealed strong overall reading instruction; 
nevertheless, no observed class sessions offered more than four minutes of vocabulary 
instruction or practice. Although Truckmiller et al. (2019) concluded that more teachers should 
be trained in academic language instruction, the extent to which special education training 
programs include this type of instruction is unknown. 

Importantly, students in CHAAOS classes maintained knowledge of the taught words four 


weeks after the conclusion of the intervention. This finding is noteworthy because CHAAOS is 
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of longer duration with a larger core of vocabulary than most vocabulary studies in special 
education classes; in the current study students were taught 48 academic words across twelve 
weeks of instruction spaced across seven months. By contrast, most vocabulary instructional 
studies in special education have been of short duration (Bos & Anders, 1990) or focused on a 
content-specific and smaller set of words (Kennedy et al., 2015; Seifert & Espin, 2012; Swanson 
et al., 2016). For students to learn as many grade appropriate academic words and maintain them 
over time is a rare finding in the special education literature. 

We note that students in BAU special education ELA classes also grew in knowledge of 
these academic words over time. Words taught during the CHAAOS intervention were drawn 
from widely available sources, including recommended words on Coxhead’s (2000) and 
Common Core (National Governors Association, 2010) lists, as well as the school district’s list 
for eighth grade general education ELA. The CHAAOS list of words was available in all classes 
because we had provided the intervention word lists to teachers and administrators in both 
conditions in all three years of the CHAAOS intervention. Nevertheless, the rate of growth in the 
BAU condition was slow in relation to students in CHAAOS classes. Taken together, these 
findings demonstrate the need for systematic vocabulary intervention to improve this key 
component of reading development for students with LD, regardless of first language. 
Generalized Vocabulary and Reading Comprehension Outcomes and Measures 

Few studies in general education (e.g., Lawrence et al., 2015; Lesaux et al., 2010, 
McKeown et al., 2018), special education (e.g., Bos & Anders, 1990; Jitendra et al., 2004), or 
with EL populations (e.g., Carlo et al., 2004; Lesaux et al., 2010) demonstrate improvement on 
standardized measures of vocabulary or comprehension as a result of vocabulary intervention. 


Although theoretically disconcerting given vocabulary’s role in reading comprehension (Kintsch, 
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1988; Perfetti & Stafura, 2014), several factors contribute to the scarcity of measureable 
generalized gains linked to vocabulary intervention. 

First, interventions and standardized measures barely touch the range of vocabulary that 
supports reading comprehension at any particular grade level. We attempted to improve over 
earlier studies by including 48 academic words that were recommended for instruction to 8" 
grade students. Since this study reports the third year of intervention, several participants in 
CHAAOS classes had instruction on up to 144 words by the close of eighth grade. Second, the 
standardized measures we used (i.e., the CREVT and WJ-III) were broad range measures with 
ceiling criteria during test administration that prohibited most students from participating in the 
range of items intended to sample their grade level. Third, we intentionally excluded from our 
instructional lists words tested directly on the CREVT to avoid bias toward the treatment. 

Thus, we were surprised to find significant differences between conditions on the 
CREVT favoring students in CHAAOS classes. However, despite evidence that students in the 
CHAAOS classes made significant gains on this transfer measure of vocabulary knowledge 
compared to students in BAU, standardized scores on the CREVT do not demonstrate growth 
against the test norming group of students in general education. Rather, students in BAU 
conditions fell farther behind on the CREVT and students in CHAAOS showed less slippage in 
scores over time, a finding also reported by Vaughn et al. (2012). 

Despite improving vocabulary, students in CHAAOS classes did not indicate significant 
growth on the reading comprehension standardized measure (WJ-III) relative to students in 
BAU. Even so, their scores did not decline significantly over time, which is common for 
students with disabilities in middle school (Vaughn et al., 2012). The limited range of items for 


each grade level on the WJ-III may have made it difficult to show growth based on exposure to 
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just 48 new words, and especially on words that may have contributed to reading comprehension 
at higher levels than students could reach before exceeding the test ceiling. 

We see a related issue in the low-to-moderate validity ratings of our researcher-developed 
measures of vocabulary with the CREVT and WJ reading comprehension (i.e., validity ranges 
.30-.68, with most below .5). Students learned the taught words with CHAAOS instruction; 
however, their overall reading ability remained in the second-to-third grade range. Thus high 
scores on learning taught words linked poorly with overall reading ability. Interestingly, the 
highest validity with standardized scores was drawn from the measure of student-generated 
definitions and sentences, which indicates a higher level of learning than selecting definitions 
(Krathwohl, 2002). It is possible that students who performed well on these production tasks 
may also have been able to answer more items correctly on the standardized measures. 
Limitations 

This study explored the potential of CHAAOS intervention to improve the vocabulary of 
students with disabilities in middle school who may also be EL. It was conducted in just one 
school district with the eighth grade students who received special education in three schools; 
thus we lack evidence for broad generalization of findings. With a larger sample, it may have 
been possible to analyze the cumulative effects of intervention; however, mobility in this district 
reduced student numbers, which only allowed analysis of the current year’s intervention effects. 
Thus, it is impossible to determine whether the effects reported here are due to eighth grade 
intervention alone, or whether intervention in prior years had some influence on student 
outcomes. Small sample size also inhibited more sophisticated analytic methods. 

We were interested in whether students who were EL would learn the words as well as 


students who were NES. Even though we found no significant differences based on first 


Vocabulary Intervention 25 


language, two concerns limit support for the hypothesis that students who were EL learned as 
much in CHAAOS classes as did students who were NES. First, the number of NES was smaller 
in both conditions than students who were EL; thus a non-significant finding could be due to 
insufficient power in the analysis. Moreover, all of the participants who were EL spoke Spanish 
as their first language, and so no conclusion can be drawn regarding speakers of other languages. 
Implications 

In CHAAOS classes, special education teachers devoted fewer than 20 minutes of their 
ELA allotted time to explicit and interactive vocabulary instruction. Although teaching 
CHAAOS decreased available time for other important instruction, devoting time to vocabulary 
did not lead to students falling farther behind in reading comprehension. Moreover, students 
maintained meanings of words they learned through CHAAOS instruction four weeks after 
instruction ended, which suggests that teaching vocabulary can be a valuable use of class time. 

The current research is important in being one of few studies to focus on academic 

vocabulary instruction for the growing population of students with LD and other disabilities who 
are also EL in middle school. This study demonstrates that when effective vocabulary practices 
that have been validated for students in general education, students with LD, or students who are 
EL are implemented (e.g., explicit instruction on word meanings, frequent exposure to the words 
in the classroom, extensive practice, review, and time spent discussing the words), learning is 
strong for students who have LD and are also EL. Therefore, we suggest that academic 
vocabulary instruction on grade appropriate words is feasible in special education middle school 
classes in which most students read below their grade level. Special education teachers 
implemented CHAAOS with high fidelity and materials are available readily to download by 


teachers and researchers to evaluate replicability. 
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Table 1 


Demographic Data 


Business As Usual Treatment Total 
Variable (n= 17) (n = 36) (n = 53) 
% n % n % n 
Gender 
Male 58.8% 10 77.8% 28 71.7% 38 
Female 41.2% 7 22.2% 8 28.3% 15 


Special Education Classification 


SLD 70.6% 12 80.6% 29 77. 4% 41 
Autism 5.9% 1 8.3% 3 7.5% 4 
OHI 17.6% 3 5.6% 2 9.4% 5 
SLI 5.9% al 2.8% 1 3.8% 2 
ED 0% 0 2.8% 1 1.9% 1 
Ethnicity 
Hispanic 100% iW 77.8% 28 84.9% 45 
Caucasian 0% 0 16.7% 6 11.3% 6 
African 0% 0 5.6% Z 3.8% 2 
American 
Missing 4.0% 1 0% 0 1.9% 1 


Language Preference 
Spanish 88.2% 15 63.9% 23 71.7% 38 


English 11.8% 2 36.1% 13 28.3% 15 


Table 2 


Scores on Taught Vocabulary by Treatment Condition 
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Measure Business as Usual Treatment 
(n = 17) (n = 36) 
Pretest Posttest Pretest Posttest 
Word Knowledge 
Cycle 1 Multiple Choice 6.56 (2.71) 6.87 (2.39)* 8.09 (3.04) = 14.37 (2.83)* 
Cycle 2 Multiple Choice 6.38 (1.89)  6.44(1.50)* 7.69(3.22) 14.18 (2.92)* 
Cycle 3 Multiple Choice 6.69 (2.09) 7.31 (2.94)* 7.23 (2.43)* 13.91 (3.17)* 
Cycle 1 Student Definitions 2.63(3.10)  4.00(4.68)* 5.26(3.82) 15.50 (8.90)* 
Cycle 2 Student Definitions 2.13(2.73)  3.56(2.94)* 2.68(2.59) 11.21 (7.40)* 
Cycle 3 Student Definitions 0.56(1.21) 1.63(3.14)* 1.21(1.79) 8.06 (6.87)* 
Using Words in Context 
Cycle 1 Usage 3.81 (1.52)  3.93(1.58)  3.86(1.77) 5.51 (2.09) 
Cycle 2 Usage 2.81 (1.83) 3.38 (1.71)* 3.97 (2.20) 5.09 (2.11)* 
Cycle 3 Usage 3.13 (1.41)  3.50(1.51)* 4.20(1.49) 5.09 (1.72)* 
Cycle 1 Student Sentences 3.69(3.42) 4.67 (5.19)* 5.47 (3.86) = 12.29 (8.23)* 
Cycle 2 Student Sentences 2.69(3.20) 4.81 (4.72)* 3.56(2.80) 10.12 (6.71)* 
Cycle 3 Student Sentences 0.75 (1.13) 2.19(2.51) 1.44(1.73) — 6.76 (5.55) 
Maintenance Pre Post Follow-up Pre Post Follow-up 
Mean 6.00 7,00" Vea" 5.81 16.38* 13.97" 
Standard Deviation (1.89) (1.64) (2.45) (3.16) (2.97) (3.61) 


Note: Significant differences are marked with asterisks. 
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Table 3 


Scores on Standardized Measures by Treatment Condition 


Measure Business as Usual CHAAOS Treatment 


(n= 17) (n = 36) 


CREVT General Vocabulary Index 
Time 1 74.89 (7.61) 74.03 (8.93) 
Year 3 Spring 69.20 (8.23)* 72.97 (6.30)* 
W/J-III Passage Comprehension 
Time 1 55.94 (13.93) 58.37 (12.26) 


Year 3 Spring 54.73 (12.85) 61.07 (11.90) 


Note: CREVT = Comprehensive receptive and Expressive Vocabulary Test. WJ-III = 
Woodcock Johnson, 3" edition. Time 1 = First administration of this measure, which was Fall of 
Year 1 for most students, or as students entered the study by moving into BAU or CHAAOS 


classes. Significant differences are marked with asterisks. 
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Table 4 


Treatment Outcomes by Language Status 


Measure Treated English Learners Treated Native English 
(n = 23) Speakers 
(n= 11) 
Pretest Posttest Pretest Posttest 
Word Knowledge 


All Cycles Multiple Choice 21.52 (6.69) 42.35 (8.75)  26.00(7.39) 42.73 (6.80) 

All Cycles Student Definitions 7.78 (6.73) 30.00 (21.16) 11.36(8.62) 42.36 (20.36) 
Using Words in Context 

All Cycles Usage 10.74 (3.68) 14.35 (5.01) 14.36(3.56) 18.00 (2.86) 


All Cycles Student Sentences 8.39(5.84)  23.87(17.61) 14.00(8.14) 38.00 (16.44) 


